Google Search Advocate John Mueller has offered advice on handling legacy AMP subdomains and their effect on the crawl budget, addressing a frequent concern among large website owners.
A website owner managing around 500,000 URLs sought guidance on how to handle crawling on an abandoned AMP subdomain.
He mentioned that crawling continues despite having implemented 301 redirects three years prior.
This situation highlights the challenges websites encounter when moving away from previously used AMP solutions.
Key Details
The site owner reported that Googlebot is still crawling hundreds of thousands of AMP URLs, although:
- All AMP URLs were redirected to the main domain three years ago
- The AMP sitemap file is empty
- Both HTTP and HTTPS versions are impacted
Mueller’s Response
Mueller proposed two simple solutions:
- Continue with the current redirect setup
- Remove the hostname from DNS entirely
Mueller played down concerns regarding the crawl budget impact by noting:
“If it’s a separate subdomain, it’ll generally have its own ‘crawl budget’. Also, with 500k pages, I don’t know if you’d really need to worry about crawl budget, even if it were not on a separate host name.”
Context
This advice comes at a time when many publishers are reevaluating their AMP implementation strategies.
Mueller’s comments suggest that complex technical fixes may not be required to manage outdated AMP content.
For website owners facing similar situations, Mueller pointed to Google’s guide on managing crawl budget for large sites for further details.
- Also see: How To Manage Crawl Budget For Large Sites
Analysis
Mueller’s response implies that optimizing the crawl budget is a valid concern for huge websites.
However, for mid-sized sites with about 500,000 pages, it may not be a significant issue.
Next Steps
This guidance provides a path forward for technical SEO professionals dealing with the shift from AMP implementations.
When addressing legacy AMP implementations, consider:
- Maintaining existing 301 redirects
- Implementing a DNS-level solution
- Consulting Google’s crawl budget documentation for specific cases
